Generating optimal strategy for providing offers

ABSTRACT

Generating optimal strategies for providing offers to a plurality of customers is described. A plurality of categorical attributes (for example, gender and residential status) and ordinal attributes (for example, risk score and credit line utilization) can be determined. Values of one of more categorical attributes can be changed as per a transition probability table. Some probabilities can be varied to determine a first tradeoff, based on which a first updated strategy can be generated. Further, noise can be added to one or more ordinal attributes. Standard deviation of a noise distribution associated with the noise can be varied so as to determine a second tradeoff, based on which a second updated strategy can be generated. The second updated strategy can be an update of the first updated strategy. Offers can be provided to the plurality of customers in accordance with the second updated strategy.

RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 61/513,392, filed on Jul. 29, 2011, and entitled “Data-DrivenLearning Of Utility-Maximizing Customer Treatment Policies,” content ofwhich is incorporated herein by reference in entirety.

TECHNICAL FIELD

The subject matter described herein relates to generating optimalstrategies for providing offers to a plurality of customers.

BACKGROUND

Conventionally, business entities provide offers to customers. Forexample, banks provide credit line increases to existing credit cardcustomers. Typically, a fixed (that is, not changing with time)strategy, such as a fixed decision tree, is used to decide a credit lineincrease to a particular customer. Formation of such fixed strategies(for example, fixed credit line increases) can require significantmanual expertise from analysts and domain experts, thereby requiring alot of time, effort, and associated costs.

SUMMARY

The current subject matter describes generating optimal strategies forproviding offers to a plurality of customers. In one aspect, dataassociated with a plurality of individuals can be obtained. Eachindividual can be associated with a plurality of attributes. Using theobtained data, a best offer for each attribute can be determined. Adecision tree characterizing best offers for corresponding attributescan be formed. Performance of the decision tree can be compared withperformance of a challenger decision tree to obtain a best performingdecision tree. Offers can be provided to the plurality of individuals inaccordance with the best performing decision tree.

In some variations, a plurality of possible offers for the plurality ofindividuals can be determined based on the obtained data. The bestoffers for each attribute can be selected from the plurality of possibleoffers.

Using the obtained data, a plurality of causal models can be determined.By joining two or more causal models, a decision model that evaluatesone or more objectives of an entity can be formed. The causal model canbe used to determine the best offer for each attribute. The causalmodels can characterize a response of an individual to a historicaloffer. The determining of the best offer can be based on evaluation ofat least one of a global maximum value and a local maximum value by thedecision model.

The challenger decision trees can be obtained by changing values of someattributes associated with the decision tree. The performance of thedecision tree can be characterized by business efficacy provided byimplementing a strategy associated with the decision tree. The businessefficacy associated with the decision tree can be based on a pluralityof iterations of strategy evolution. The performance of the challengerdecision tree can be characterized by business efficacy provided byimplementing a strategy associated with the challenger decision tree.The business efficacy associated with the challenger decision tree canbe based on a plurality of iterations of strategy evolution. Thebusiness efficacy can characterize a profit of an entity providing theoffers to the plurality of individuals. The performance of the decisiontree can be characterized prior to implementation by expected futurebusiness efficacy based on the decision model, and the performance ofthe challenger decision tree can be characterized prior toimplementation by expected future business efficacy based on thedecision model. The expected future business efficacy can be simulatedbased on multiple iterations of future strategy evolution.

In another aspect, a graph characterizing a strategy for providingoffers can be obtained. One or more attributes associated with aplurality of individuals can be modified. The one or attributes can berepresented by the graph. Based on the modified one or more attributes,an updated strategy for providing offers can be generated.

In some variations, the graph can include a plurality of dots having atleast one of a corresponding color and a corresponding intensity. The atleast one of the corresponding color and the corresponding intensity cancharacterize a value of at least one attribute for an associatedindividual. The offers can be provided to the plurality of individuals.The modifying of the one or more attributes can comprise adding noise tothe attributes. The adding of the noise to the attributes can comprisevarying a standard deviation of a noise distribution to determine thenoise. The adding of the noise to the attributes can provide an optimalprofit to an entity providing the offers. The optimal profit can be morethan a profit obtained without the addition of the noise.

In yet another aspect, attributes associated with a strategy can bedetermined. Gaussian noise can be added to one or more attributes.Standard deviation of the Gaussian noise can be varied to determine atradeoff Δn updated strategy associated with the tradeoff can begenerated.

In some variations, offers can be provided based on the updatedstrategy. The updated strategy can be determined based on the tradeoff.The tradeoff can characterize a balance between cost of a businessentity and rate of update of strategies.

In a further aspect, a plurality of attributes associated with astrategy can be determined. Values of a first set of one or moreattributes can be changed in accordance with a transition probabilitytable. One or more probabilities can be varied to determine a firsttradeoff. Based on the first tradeoff, a first updated strategy can begenerated.

In some variations, noise can be added to a second set of one or moreattributes. Standard deviation of a noise distribution associated withthe noise can be varied to determine a second tradeoff. Based on thesecond tradeoff, a second updated strategy can be generated. The secondupdated strategy can characterize an update of the first updatedstrategy. Based on the second updated strategy, offers can be providedto a plurality of individuals. The first set of one or more attributescan include gender and residential status. The second set of one or moreattributes can include risk score and credit line utilization.

Further, from a table, eligibility constraints can be determined forprovision of one or more offers to one or more customers. The firstupdated strategy and the second updated strategy can be based on theeligibility constraints to exclude provision of some offers tocorresponding ineligible customers.

The first tradeoff and the second tradeoff can be determined usingcorresponding tradeoff curves. Each tradeoff can be characterized by asweet-spot on a corresponding tradeoff curve. The sweet-spot cancharacterize a position where generated strategy data can be more than afirst threshold while profit can be more than a second threshold.

Non-transitory computer program products are also described thatcomprise instructions, which, when executed by one or more dataprocessors, causes at least one data processor to perform operationsherein. Similarly, computer systems are also described that may includea processor and a memory coupled to the processor. The memory maytemporarily or permanently store one or more programs that cause theprocessor to perform one or more of the operations described herein.Methods can be implemented by one or more data processors forming partof one or more computing systems.

The subject matter described herein provides many advantages. Forexample, using the current subject matter, generating strategies toprovide offers to customers can require minimal, negligible or no manualexpertise by analysts and domain experts, thereby requiring less time,effort, and associated costs. Further, the effectiveness of a strategycan be shown, for a better visual analysis of the strategy, on a graph(or other model) displayed on a graphic user interface. Thiseffectiveness of the strategy can be changed by varying at least onestatistical parameter, such as standard deviation, so as to generate amost optimal strategy that can provide maximum business efficacy.Champion-challenger techniques can also be implemented in some otherimplementations.

The details of one or more variations of the subject matter describedherein are set forth in the accompanying drawings and the descriptionbelow. Other features and advantages of the subject matter describedherein will be apparent from the description and drawings, and from theclaims.

DESCRIPTION OF DRAWINGS

FIG. 1A is a first flowchart illustrating an ongoing provision ofoptimal offers to a plurality of customers;

FIG. 1B is a second flowchart illustrating an ongoing provision ofoptimal offers to a plurality of customers;

FIG. 1C is a third flowchart illustrating an ongoing provision ofoptimal offers to a plurality of customers;

FIG. 1D is a fourth flowchart illustrating an ongoing provision ofoptimal offers to a plurality of customers;

FIG. 2 is a diagram illustrating a decision tree;

FIG. 3 is a plot illustrating an offer provided to each customerassociated with attributes risk score and credit line utilization, inaccordance with a champion strategy of providing offers;

FIG. 4 is a plot illustrating an offer provided to each customerassociated with attributes risk score and credit line utilization, inaccordance with a challenger strategy of providing offers;

FIG. 5 is a plot illustrating an offer provided to each customerassociated with attributes risk score and credit line utilization, inaccordance with a champion-challenger test that can provide morecommon-support regions;

FIG. 6 is a diagram illustrating a system for varying attributes in acontrolled manner so as to control the area of common support regions ina plot associated with those attributes;

FIG. 7 is a plot illustrating an offer provided to each customerassociated with attributes modified risk score and modified credit lineutilization that can provide even more and better controlled commonsupport regions than those with respect to FIG. 5;

FIG. 8 is a table illustrating offer eligibilities of a plurality ofcustomers;

FIG. 9 is a plot illustrating an offer provided to each customerassociated with attributes risk score and credit line utilization, afterthe offer eligibilities have been imposed to either provide or denyoffers to corresponding customers;

FIG. 10 is a plot illustrating an offer provided to each customerassociated with attributes risk score and credit line utilization, inaccordance with a true optimal strategy;

FIG. 11 is a diagram illustrating changes in strategies for providingoffers and associated changes in business efficacy with differentiterations of strategy optimization, in accordance with a timidtesting/modeling technique;

FIG. 12 is a diagram illustrating changes in strategies for providingoffers and associated changes in business efficacy with differentiterations of strategy optimization, in accordance with an aggressivetesting/modeling technique;

FIG. 13 is a diagram illustrating an evaluation system that candetermine the exploration-versus-exploitation-tradeoff betweenaggressiveness and timidness; and

FIG. 14 is a diagram illustrating a graph that can be displayed todetermine the exploration-versus-exploitation-tradeoff.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

The current subject matter relates to generating optimal strategies forproviding offers to a plurality of customers. The strategies can beillustrated using strategy tools, such as decision trees. While decisiontrees have been described herein, other strategy tools can also be used,such as one or more of the following: decision tables, flow-charts,if-then-else analyses, switch-case analyses, what-if analyses, influencediagrams, Markov chains, odds algorithms, truth tables, and any othertool.

In an example of credit card limit management, an offer can be anincrease in credit card limit associated with a customer. A firstcustomer, which has a very low credit bureau score (that is, highlikelihood of defaulting in the near future), may not be provided anoffer or increase in credit limit. A second customer, which has a highercredit score and uses the credit score once in a while for minorpurchases, can be provided a small (for example, $100-$500) increase incredit card limit. A third customer, which has a very high risk scoreand has account balance close to current limit, can be provided a large(for example, $1000-$10000) increase in credit card limit. Such adifferentiation in provision of offers can be advantageous for a lenderthat may want to maximize revenue (for example, through card interest orinterchange fees) while limiting exposure and delinquencies.

Furthermore, in an example of retail coupon marketing, an offer can be adiscount coupon associated with a purchase of one or more items, such asretail products. A loyal customer that may be expected to beprice-sensitive can be offered a discount coupon for purchasing ahigher-priced brand, whereas another customer that is expected to beless price-sensitive may not be offered the discount. Price-sensitivitycan be inferred from historic purchase transactions (of associatedcustomer) that can reveal strong preferences for discounted products.Such a differentiation in provision of offers can be advantageous for aretailer that may want to reward loyalty and increase sales revenue, orcan be advantageous for a manufacturer that may want to increase marketshare.

FIG. 1A is a first flowchart 100 illustrating an ongoing provision ofoptimal offers to a plurality of customers. The provision can result ina currently most optimal utilization of resources of an entity. The mostoptimal utilization of resources can include less manual work, lesseffort, less time, less cost, higher profit, less computingrequirements, less storage requirements, and optimal use of otherbusiness resources.

Data can be obtained, at 102, for a plurality of customers. The data caninclude historic information associated with various attributes ofcustomers. The attributes can include observed variables that can beknown at the time of provision of offers, such observed variablesincluding: information filled by customers in applications, informationregarding financial-accounts, demographics, transaction data, creditbureau data, credit card score, credit card usage data, risk score,revenue score, credit line utilization data, social network data,conversations/messages of one or more customers with other customers orthird parties, third party data, and any other observed variable.Further, the attributes can include derived or inferred variables, suchderived variables including: text keywords, n-grams, merger andacquisition transaction data, parameters of social networks such asconnectedness (for example, roll-up activity), and any other derivedvariable. Furthermore, the attributes can include forward-lookingpredictive variables and score that can be computed based on theobserved variables, the predictive variables including: likelihood todefault over a particular (for example, predetermined) period of time infuture (for example, two years), expected customer lifetime value, andany other predictive variable.

The obtained data can be used to determine, at 104, various offers thatcan be provided to a customer. The various offers can include variousincreases in credit limit for a customer. For example, different creditline increases can be at least some of: $0 (that is, no increase), $500,$1000, $2500, $5000, $10000, $50000, and any other credit line increase.The determining of offers can be determined automatically by performingregression analyses on the obtained data. Although an automaticdetermining of offers is described, in other implementations, manualdetermining of offers can also be possible.

Using the obtained data, causal models can be determined, at 106. Usingthe causal models, effects of actions on outcomes can be inferred fromthe obtained data. Thus, the causal models are also referred to hereinas action-effect models. A causal model can characterize a responsefunction, such as: “function (attribute, historical-offer)=Outcomesubsequent to the historical-offer.” In one example, thehistorical-offer can be an offer, and the outcome can be either anacceptance or denial of the offer by the customer. In another example,the historical-offer can be an offer for a $500 increase in creditlimit, and the outcome can be one or more of subsequent credit cardbalances, delinquencies, losses, and any other outcome. The causalmodels can be developed either for each outcome or for selectiveoutcomes. Such a development of causal models can be performed usingregression modeling. Extrapolation risk can be removed-from orminimized-in the causal models, as described further below.

Two or more causal models can be joined/tied together to form, at 108, adecision model. The decision model can evaluate business objectives ofan entity, such as a business entity. The business objectives cancharacterize business metrics, such as cost, profits, loss, salesvolume, revenue, and any other business metric. The decision model cancharacterize business efficacy, which can be defined by the followingexemplary equation: “Efficacy(attribute, offer)=a*M1(attribute,offer)+b*M2(attribute, offer),” wherein Efficacy can be businessefficacy, M1 can be a first business metric, M2 can be a second businessmetric, and a and b can characterize constraints associated withcorresponding business metrics. While the business efficacy equation hasbeen described as being dependent on two business metrics M1 and M2, inother implementations, more than two (for example, three, five, ten,twenty, hundred, five hundred, or any other finite number, asappropriate) metrics can be used.

A best offer for each attribute can be determined, at 110, by using thedecision model. More specifically, the best offer can be determinedusing the business efficacy equation, which is described above as:“Efficacy(attribute, offer)=a*M1(attribute, offer)+b*M2(attribute,offer),” wherein Efficacy can be business efficacy, M1 can be a firstbusiness metric, M2 can be a second business metric, and a and b cancharacterize constraints associated with corresponding business metrics.The best offer for an attribute can be one that maximizes businessefficacy. That is, the best offer for a particular attribute can be thevalue of offer that can maximize the business efficacy equation:“Efficacy(attribute, offer)=a*M1(attribute, offer)+b*M2(attribute,offer).” The obtained maximum can be a global maximum. Although use of aglobal maximum is described, in some other implementations, the obtainedmaximum can be a local maximum. Accordingly, the best offer can becharacterized by the following mathematical equation: “Offer^(Best)(Attribute)=argmax {Efficacy(attribute, offer)},” wherein Offer^(Best)(Attribute) can be the best offer for a particular attribute, and argmaxcan characterize argument of the maximum.

The determined best offers for corresponding attributes can be used toform, at 112, a decision tree. This decision tree can be called achampion decision tree. A decision tree can be a graph or a model thatcan characterize decisions and their possible consequences, such aschance outcomes, resource costs, efficacy, and the like. A decision treecan characterize an algorithm associated with a model.

Challenger decision trees can be obtained, at 114. The challengerdecision trees can be obtained by changing some values in a championdecision tree. For example, a challenger tree can be constructed bychanging some of the actions (offers) in the leaf nodes as determined bya user. Alternately, a challenger can be constructed by changing some ofthe split values to different values as determined by a user.

Performances of the champion decision tree and the challenger decisiontrees are compared, at 116. More specifically, the decision tree,provision of offers according to which can provide maximum businessefficacy/advantages, can be determined.

Offers can be provided, at 118, in accordance with the best performingtree. In a first iteration, the champion decision tree can usually havethe best performance. That is, in the first iteration, the provision ofoffers in accordance with the champion decision tree can provide mostbusiness efficacy than provision of offers in accordance with anychallenger decision tree. In subsequent iterations, if a challengerdecision tree challenges the champion decision tree and becomes thechampion decision tree, the new champion decision tree (previouslychallenger decision tree) can provide more business efficacy.

The offers can be provided to virtual customers in iterations until amost optimal strategy (for example, the most optimal decision tree thatcan provide optimal business efficacy) is obtained. After the mostoptimal strategy is obtained, the offers can be provided to realcustomers.

It can be determined, at 120, whether a threshold time haselapsed/passed since the provision of offers according to a bestperforming decision tree. The particular (for example, predetermined)time can be 5 minutes, 10 minutes, 1 hour, 2 hours, 10 hours, 1 day, 2days, 10 days, 20 days, 3 months, or any other time. A new action maynot be performed until the particular (for example, predetermined) timehas elapsed/passed. When the particular (for example, predetermined)time has elapsed, one or more (or all) of 102, 104, 106, 108, 110, 112,114, 116, 118, and 120 can be re-performed such that the most currentdata can be used to provide optimal offers to customers.

FIG. 1B is a second flowchart 130 illustrating an ongoing provision ofoptimal offers to a plurality of customers. A graph can be obtained at130. The graph can characterize a strategy of providing offers to aplurality of customers. The graph can show values of differentattributes, such as risk score and credit line utilization. One or moreattributes (for example, risk score and credit line utilization) can bemodified at 134. The modification can include addition of noise to eachattribute. The noise can be Gaussian noise. Standard deviation of theGaussian noise can be varied to determine a most optimal standarddeviation that can be used to provide most business efficacy (forexample, profit of an entity providing the offers). An updated strategyto provide offers can be generated, at 136, based on the modified one ormore attributes. Offers can be provided, at 138, based on the updatedstrategy. The offers can be provided to virtual customers in iterationsuntil a most optimal strategy (for example, the most optimal decisiontree that can provide optimal business efficacy) is obtained. After themost optimal strategy is obtained, the offers can be provided to realcustomers. It can be determined, at 140, if a threshold time haselapsed. If the threshold time has elapsed, 132, 134, 136, 138, and 140can be re-performed. If the threshold time has not elapsed, new actionmay not be performed.

FIG. 1C is a third flowchart 160 illustrating an ongoing provision ofoptimal offers to a plurality of customers. Attributes associated with astrategy for providing offers to a plurality of customers can bedetermined at 162. The attributes can be risk score, credit lineutilization, and the like. Gaussian noise with an associated Gaussiandistribution can be added, at 164, to one or more attributes (in someimplementations, all attributes). Standard deviation of the Gaussiannoise associated with each attribute can be varied, at 166, to determinean exploration-exploitation tradeoff. An updated strategy can begenerated, at 168, based on the exploration-exploitation tradeoff. Theexploration-exploitation tradeoff can maintain a balance betweenaggressiveness (so as to learn as much and as fast as possible) andtimidness (so as to keep testing costs under control). Based on theupdated strategy, offers can be provided to customers. The offers can beprovided to virtual customers in iterations until a most optimalstrategy (for example, the most optimal decision tree that can provideoptimal business efficacy) is obtained. After the most optimal strategyis obtained, the offers can be provided to real customers.

FIG. 1D is a fourth flowchart 180 illustrating an ongoing provision ofoptimal offers to a plurality of customers. Attributes associated with astrategy for providing offers to a plurality of customers can bedetermined at 182. The attributes can be categorical attributes, such asgender, residential status, and the like. Values of some attributes canbe changed, at 184, with some probability in accordance with atransition probability table. The transition probability table can beaccessed either via wires or wirelessly over a network. One or moreprobabilities in the transition probability table can be varied, at 186,to determine a tradeoff. The tradeoff can be used to generate, at 188,an updated strategy that can be more optimal than the strategy in theprevious iteration. The tradeoff can maintain a balance between a fastand adequate learning and a high cost. Based on the updated strategy,offers can be provided to customers. The offers can be provided tovirtual customers in iterations until a most optimal strategy (forexample, the most optimal decision tree that can provide optimalbusiness efficacy) is obtained. After the most optimal strategy isobtained, the offers can be provided to real customers.

FIG. 2 is a diagram illustrating a decision tree 200 in accordance withsome implementations of the current subject matter. The decision tree200 can characterize a strategy for providing offers 202 to a pluralityof customers. The offers 202 can be increases in credit limit forselective (for example, all eligible) accounts 204 of one or morecustomers of the plurality of customers. The offers 202 can be increasesin credit limit of $ zero (that is, no increase) 206, $ zero (that is,no increase) 208, $ two-thousand 210, and $ three-thousand 212. Eachcustomer can be associated with at least the following attributes: riskscore 214, revenue score 216, and credit line utilization 218.

The risk score 214 can be one of low 220, medium 222, and high 224. Thedifferentiation between low 220, medium 222, and high 224 risk scorescan be based on corresponding split values. For example, the risk score214 can be from 1 to 100, and the split values can be thirty-three andsixty-six so as to classify a particular risk score 214 as one of low220, medium 222, and high 224. If the risk score 214 is low, no offermay be made regardless of values of revenue score 216 and credit lineutilization 218.

If the risk score 214 is medium, the revenue score 216 can be classifiedinto low 226, medium 228, and high 230. The revenue score 216 can beclassified into low 226, medium 228, and high 230 based on correspondingsplit values. When the risk score 214 is medium 222 and the revenuescore is high 230, the offer can be $ two-thousand 210 increase incredit limit. If the risk score 214 is high 224, the revenue score 216can be classified into low 232, medium 234, and high 236. Thisclassification of the revenue score 216 can be based on correspondingsplit values.

In one implementation, the split values associated with low 226, medium228, and high 230 revenue scores can be same as split values associatedwith low 232, medium 234, and high 236 revenue scores. In otherimplementations, the split values associated with low 226, medium 228,and high 230 revenue scores can be different from split valuesassociated with low 232, medium 234, and high 236 revenue scores.Similar splits and corresponding offers can be made in the decision tree200, as shown.

All (or most) of the attributes 214, 216, and 218, and associated splitvalues can be determined from the historic data obtained at 102. Inother implementations, the attributes 214, 216, and 218, and/or thesplit values can be specified by a designer.

FIG. 3 is a plot 300 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes risk score 302 and credit line utilization 304, in accordancewith a champion strategy of providing offers. Each dot in the plot 300can characterize a customer. At least one of color and intensity of thedot can characterize offer (for example, credit line increase) providedto the customer. The offer can be provided in accordance with thestrategy characterized by a champion decision tree, such as the decisiontree 200. A scale 306 can characterize the quantitative numerical valueof the offer associated with a corresponding color or intensity. Thescale 306 can display a continuous range of colors and/or intensities.Thus, the scale 306 can be used to infer the numerical value of theoffer provided to a customer associated with a particular color orintensity.

The strategy associated with plot 300 can provide, among other offers,offers of $ zero (that is, no increase in credit limit), $ two-thousand,and $ five-thousand. The highest offers can be provided to customerswith moderate to high risk scores 302, and with moderate credit lineutilizations 304. For example, the highest offers (for example,increases of credit limits by $ five-thousand or about $ five-thousand)can be provided to customers with risk scores 302 being more thanseven-hundred-and-fifty, and with credit utilizations 304 between twentyand hundred. Medium offers can be provided to customers with medium riskscores 302 and high credit line utilizations 304. For example, mediumoffers (for example, increases of credit limits by $ two-thousand orabout $ two-thousand) can be provided to customers with risk scores 302between six-hundred-and-eighty and seven-hundred-and-fifty, and withcredit line utilizations 304 between seventy and hundred. No offers maybe provided (that is, credit limit is not increased) to most of theother customers, as shown.

The plot 300 includes some common regions where some customers receive afirst offer (for example, increases of credit limits by $ five-thousand)whereas other customers receive a second offer (for example, increasesof credit limits by $ two-thousand). For example, consider the regionsin plot 300 where credit utilization 304 is fifty and risk score 302 isbetween seven-hundred-and-fifty and eight-hundred. Within these commonregions, customers with similar values of attributes can receivedifferent offers.

The common regions are also referred to, herein, as common supportregions.

FIG. 4 is a plot 400 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes risk score 302 and credit line utilization 304, in accordancewith a challenger strategy of providing offers. As shown, the regionsassociated with different offers can be defined differently in plot 400from those in plot 300. For example, the challenger strategy can requirea risk score 302 of at least seven-hundred-and-eighty for a customer toreceive the highest offer, whereas the champion strategy of plot 300requires a risk score 302 of seven-hundred-and-fifty for a customer toreceive the highest offer.

The plot 400 can also include common support regions, as shown.

Common support regions in plots 300 and 400 can be common regions wheresome customers receive a first offer (for example, increases of creditlimits by $ five-thousand) whereas other customers receive a secondoffer (for example, increases of credit limits by $ two-thousand). Thus,alternate offers can be tested on some customers in these common regionsso as to maintain or increase the business efficacy of the businessentity providing these offers. In non common-support regions (that is,regions that may not have common support), the outcomes from alternativeoffers can be unknown, and therefore extending alternative offers tocustomers in these non common-support regions can carry higher risks,which a risk-averse business may want to avoid. Thus, it can beadvantageous to have/implement strategies with more (for example, morein area/size) common-support regions.

FIG. 5 is a plot 500 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes risk score 302 and credit line utilization 304, in accordancewith a champion-challenger test that can provide more (for example, morein area/size) common-support regions than those in plots 300 and 400. Inthe champion-challenger test, each customer can be assigned to eitherthe champion strategy of plot 300 or the challenger strategy of plot400. The probability for assignment to the champion strategy can be sameas the probability for assignment to the challenger strategy. Such anassignment of customers can produce the plot 500, where the commonsupport regions can be more in area/size than area/size of those inplots 300 and 400. Such common support regions are illustrated byenclosing those regions by corresponding ellipses 502, 504, and 506. Noncommon-support regions (that is, regions that may not have commonsupport) are illustrated by showing question-marks in those regions.

As noted above, alternate offers can be tested on some customers in thecommon regions enclosed by ellipses 502, 504, and 506 so as to maintainor increase the business efficacy of the business entity providingoffers. In non common-support regions (for example, regions that may nothave common support), extending alternative offers to customers in thesenon common-support regions can carry higher risks, which a risk-aversebusiness may want to avoid. Such a testing of alternate offers inselective regions (for example, common support regions) of a plot canadvantageously maintain or improve/increase business efficacy byreducing or eliminating the risk of manual judgments associated withextrapolating effect of alternate offers in the non common-supportregions.

FIG. 6 is a diagram illustrating a system 600 (and associated technique)for varying attributes in a controlled manner so as to control the areaof common support regions in a plot associated with those attributes.The system 600 can receive attributes risk score 602 and credit lineutilization 604 from obtained historic data, as in 102. A noisegenerator 606 can add noise to the attribute risk score 602 so as toobtain a modified risk score 608. A noise generator 610 can add noise tothe attribute credit line utilization 604 so as to obtain a modifiedcredit line utilization 612. The modified risk score 608 and themodified credit line utilization 604 can be used to form a decision tree614. Offers can be provided, at 616, to a plurality of customersaccording to the decision tree 614.

The noise added to the attributes risk score 602 and credit lineutilization 604 can be Gaussian noise that can have a Gaussiandistribution associated with a mean and a standard distribution. To varythe noise added, the mean and the standard distribution can be varied bya designer in a controlled manner. The standard deviation can be aparameter that can control the spread of noise. Also, the area ofcommon-support region can be directly proportional to the standarddeviation (that is, higher the standard deviation, more thecommon-support region). So, in some implementations, the mean can be setto zero, and the standard deviation can be varied until a decision-treeand associated plot with sufficient common support are obtained.Increasing the standard deviation by a significant amount (for example,more than a predetermined threshold) can reduce business efficacy—so, abalance can be obtained to provide high (for example, more than athreshold) common-support area paired with high business efficacy.

While examples of Gaussian noise have been described, other noises thatcan have a distribution can also be used, such as Bayesian noise,Poisson noise, Cauchy noise, Brownian noise, and any other noise thatcan have a distribution. The distributions of these noises can becontrolled using one or more statistical parameters, such as standarddeviation.

FIG. 7 is a plot 700 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes modified risk score 702 and modified credit line utilization704 that can provide even more and better controlled common supportregions than those with respect to FIG. 5. The modified risk score 702can be obtained by adding, by noise generator 606, noise in a controlledmanner (for example, by varying standard deviation of the noisedistribution) to the risk score 302. The modified credit lineutilization 704 can be obtained by adding, by noise generator 610, noisein a controlled manner (for example, by varying standard deviation ofthe noise distribution) to the credit line utilization 304. As shown,area of the common support region in plot 700 can be significantly morethan the common support regions in plots 300 and 400, or even 500.

In some other implementations, the attributes can be such that additionof noise may not be appropriate. For example, when the attributes can becategorical attributes such as gender and residential status, theaddition of noise to these attributes may not be appropriate. In suchcases, a methodology that can be an alternate of the noise-additionmethodology can be implemented. In this alternate methodology, values ofsuch categorical attributes can be randomly changed/flipped according toa table, such as a transition probability table. For example, if valueof attribute residential status is “lives with parents,” then this valuecan be changed/flipped with 20% probability to the value “renter,” andvice versa, in accordance with the transition probability table providedby the designer. Further, if value of the attribute gender is “male,”then this value can be changed/flipped with 10% probability to “female,”and vice versa, in accordance with the transition probability tableprovided by the designer. Thus, while the test designer can provide avalue of standard deviation to control addition of noise in some otherimplementations described herein, here, the test designer can providethe transition probability table.

In some implementations, the probabilities associated with thetransition probability table can be uniform so that each value of eachcategorical attribute can be equally likely be flipped to any othervalue. In other implementations, the probabilities can be chosen suchthat certain value flips can be more likely than others, especially whenthere may be a sense of closeness of values of the attributes. Forexample, customers that live with their parents can be regarded moresimilar to renters and less similar to home owners. In this case, thetransition probability between values “renter” and “lives with parents”can be higher than transition probability between values “lives withparents” and “owner.” Accordingly, this can result in a decision treethat can more often consider a customer with value “lives with parents”as having a value “renter,” and vice versa, and less often consider thecustomer as having the value “owner.” Further, the transitionprobabilities may not need to be symmetric—for example, the value “liveswith parents” can be considered similar to the value “renter”treatments, but the value “renter” may be considered completelydifferent (that is, not similar) to the value “lives with parents.”

Above, (1) addition of noise, and (2) random flipping of values ofattributes using a transition probability table are described asseparate implementations. However, when customers may be associated withboth ordinal attributes (for example, risk score and credit lineutilization) and categorical attributes (for example, gender andresidential status) are used, the above-noted two methods (that is, (1)addition of noise, and (2) random flipping of values of attributes usinga transition probability table) can be used in combination. Thecombination of these two methods can be either in any sequence or inparallel.

FIG. 8 is a table 800 illustrating offer eligibilities of a plurality ofcustomers. These offer eligibilities can be used to imposelimits/constraints during provision of offers to the customers sofurther improve/increase the business efficacy. The table 800 includesoffers 802, such as first offer (A) 804, second offer (B) 806, and thirdoffer (C) 808. The offers 800 can be for customers 810, such as X1, X2,X3, X4, and so on. The table 800 further includes treatment eligibilitysets 812. The entries “1” can characterize eligible customer-offercombinations, and the entries “0” can characterize ineligiblecustomer-offer combinations. The values of offer eligibility sets 812can characterize offers 802 for which an associated customer may beeligible.

The offer eligibilities of table 800 can be determined as follows. Foreach value of an offer, customers can be determined that fall into aregion, which can be associated with attributes “X” and where this valuecan be well represented. A conditional multinomial regression model canbe implemented to predict the probability for each offer “k” as afunction of the “X.” That is, pk(X)=Probability{Offer=k|X}, wherein k=0,1, 2, . . . . A customer with attributes “X” can be eligible for offer“k” if pk(X)>c, where “c” can be a probability threshold, such as zeropoint two (that is, 0.2). Otherwise, the customer can be ineligible foroffer “k.”

FIG. 9 is a plot 900 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes risk score 902 and credit line utilization 904, after theoffer eligibilities have been imposed to either provide or deny offersto corresponding customers. The associated probability threshold “c” canbe zero point two (that is, 0.2). The offer eligibilities can beobtained from offer eligibility tables, such as table 800. The riskscore 902 can be same as either risk score 302 or modified risk score702. The credit line utilization 904 can be same as either credit lineutilization 304 or modified credit line utilization 704. Each customercan be associated with one of seven non-empty offer eligibility sets,which can be differentiated by at least one of color and intensity ofthe illustrated dots, wherein each dot can characterizes a customer.

FIG. 10 is a plot 1000 illustrating an offer (for example, credit lineincrease (CLI)) provided to each customer that is associated withattributes risk score 1002 and credit line utilization 1004, inaccordance with a true optimal strategy. This true optimal strategy canbe based on a simulation model, and is shown here for comparison andbenchmarking. The true optimal strategy can be even more optimal (forexample, provide more business efficacy) than the initial championstrategy. The champion strategy can be obtained in a first iteration offlowchart 100, and in subsequent iterations, increasingly optimalstrategies for providing offers can be obtained. After a thresholdnumber of iterations, a most optimal strategy can be obtained. The riskscore 1002 can be same as either risk score 302 or modified risk score702. The credit line utilization 1004 can be same as either credit lineutilization 304 or modified credit line utilization 704.

FIG. 11 is a diagram illustrating changes in strategies for providingoffers and associated changes in business efficacy (for example, profitof an entity providing offers) with different iterations of strategyoptimization, in accordance with a timid testing/modeling technique. Inthe timid modeling technique, standard deviation of noise added (forexample, addition of noise is described with respect to system 600) toattributes can be small. The initial strategy 1102 resembles thechampion strategy, and subsequent changes from plot to plot (e.g. 1002to 1004, 1004 to 1006, etc.) can be all small, because only a smallfraction of customers located close to decision boundaries can receivealternate treatments, as there is a small common-support region.

The standard deviations can be fixed for all iterations. In otherimplementations, the standard deviation can vary in each iteration. Forexample, the standard deviation can either increase or decrease, eitherlinearly or non-linearly, in a particular (for example, predetermined)fashion.

Further, a graph 1150 illustrates business efficacy over fiveiterations. The business efficacy can be characterized by profits andassociated costs. Line 1152 illustrates profits (in units of dollars percustomer account per year (p.p.a)) made by the test designs over fiveiterations. Line 1154 illustrates costs of testing, wherein cost oftesting can characterize a comparison between expected profit from achampion strategy and expected profit from a current test design (forexample, scattered plots 1102, 1104, 1106, 1108, and 1110) around thechampion strategy. Line 1156 illustrates benchmark profits generated bythe true optimal strategy. Lines 1152, 1154, and 1156 can have differentcorresponding colors and/or intensities for a clear visualization.

FIG. 12 is a diagram illustrating changes in strategies for providingoffers and associated changes in business efficacy (for example, profitof an entity providing offers) with different iterations of strategyoptimization, in accordance with an aggressive testing/modelingtechnique. In the aggressive modeling technique, standard deviation ofnoise added (for example, addition of noise is described with respect tosystem 600) to attributes can be increased by large values. The initialplot 1202 in the first iteration can be significantly different from theplot 1102 and from the champion strategy, because a large fraction ofcustomers located close to decision boundaries can receive alternatetreatments, as there is a large common-support region.

The standard deviations can be fixed for all iterations. In otherimplementations, the standard deviation can vary in each iteration. Forexample, the standard deviation can either increase or decrease, eitherlinearly or non-linearly, in a particular (for example, predetermined)fashion.

Further, a graph 1250 illustrates business efficacy (for example, profitof an entity providing offers) over five iterations.

The graphs 1150 and 1250 can be compared as follows. Initially, thetimid test design can make a profit of $40 per account per customeraccount per annum (p.a.a.), which can be higher than the $30 p.a.a.profit made by the moderately aggressive test design. But after a smallnumber of iterations, the moderately aggressive testing can make thehigher profit. After five iterations, the moderately aggressive testingcan makes a profit of $88 p.a.a. versus $66 p.a.a. for timid testing.

Cost of testing can characterize a comparison between expected profitfrom a champion strategy and expected profit from a current test design(for example, scattered plots 1102, 1104, 1106, 1108, 1110, 1202, 1204,1206, 1208, and 1210, whichever is current and appropriate) around thechampion strategy. For each iteration, costs of testing can also beevaluated by comparing the profit performance of the current championstrategy with the profit performance of the current test design. Costsof testing can be very small (for example, $1 p.p.a.) for timid testing,whereas the costs of testing can remain moderate (for example, $10p.p.a. or less) for the moderately aggressive testing. Further, the costof testing can shrink over time for the moderately aggressive testdesign, which can be partly due to the decreasing value of standarddeviation over time. The profit figures shown for the test designs havebeen already accounted-for in the testing costs.

Thus, as noted above, standard deviation of the noise distributionassociated with noise added to attributes can be increased to follow amore aggressive strategy. However, when the standard deviation isincreased, the cost can increase, thereby decreasing the profit.Therefore, an optimal standard deviation is used such that the profitincreases, as desired, in the iterations. Accordingly, very timid testdesigns can be rejected due to associated slow learning, and veryaggressive test designs can be rejected due to associated high costs. Toobtain a tradeoff (also referred to herein as anexploration-versus-exploitation-tradeoff) between aggressiveness (so asto learn as much and as fast as possible) and timidness (so as to keeptesting costs under control), an evaluation system can be used that candisplay and determine the tradeoff.

FIG. 13 is a diagram illustrating an evaluation system 1300 that candetermine the exploration-versus-exploitation-tradeoff betweenaggressiveness (so as to learn as much and as fast as possible) andtimidness (so as to keep testing costs under control).

The evaluation system 1300 can include a business objectives evaluator1302 that can determine an effectiveness of exploitation. To determinethe effectiveness of exploitation, the business objective evaluator 1302can determine metrics for exploitation, such as profit per customer asexpected from a test design (for example, one of scatter plots 1102,1104, 1106, 1108, 1110, 1202, 1204, 1206, 1208, and 1210, whichever isappropriate). While profit is described as a metric, other metrics canalso be used, such as loss, cost, revenue, volume, and any otherbusiness exploitation metric.

The evaluation system 1300 can further include a common supportevaluator 1304 that can determine an effectiveness of exploration. Todetermine the effectiveness of exploration, the common support evaluator1304 can determine an exploration index (EI), which can characterize anextent of common support regions in scattered plots (for example,scatter plots 1102, 1104, 1106, 1108, 1110, 1202, 1204, 1206, 1208,1210, and the like) of test designs. The exploration index and thecommon support regions can characterize amount of information capturedin the test data for developing a causal model for determining effectsof treatments on customers. Although an exploration index has beendescribed, other information regarding data support for causal modeldevelopment can also be used.

The exploration index can include a portion of customers associated withoffer eligibility sets of cardinality greater than or equal to two. Theoffer eligibility sets can be offer eligibility sets 812 in table 800.In one variation, the exploration index can calculate a sum ofcardinalities of treatment eligibility sets for all customers. Such acalculation can be advantageous for customers with larger eligibilitysets, as more causal effects can be determined and used, due to presenceof a larger dataset.

The exploration index can also be formed using different optimizationcriteria. For example, a parametric structure for quadratic causalmodels with first order interactions can be assumed. The parametricstructure can specify a design matrix for a regression model. Then, adesign optimality criterion, such as D-optimality, can be used. For thedesign optimality criterion, a determinant of the information matrix ofthe design can be calculated. The value of the determinant can bedirectly proportional to accuracy of estimation of the model parameterscharacterized by the exploration index.

The business metrics, as determined by the business objective evaluator1302, and the exploration index, as determined by the common supportevaluator 1304, can be used to find a balance (for example,exploration-versus-exploitation-tradeoff) between exploitation andexploration. Information associated withexploration-versus-exploitation-tradeoff can be used to determine offersthat can be provided to customers. These offers can be used by thebusiness objectives evaluator 1302 and the common support evaluator 1304to re-determine the effectiveness of exploitation and exploration, anddetermine new offers that can be more optimal than previous offers.

FIG. 14 is a diagram illustrating a graph 1400 that can be displayed todetermine the exploration-versus-exploitation-tradeoff. The graph 1400can include a plot of profit 1402 versus exploration index (EI) 1404.The profit 1402 can characterize exploitation, and the exploration index1404 can characterize exploration. The graph 1400 can include a tradeoffcurve 1406 that can be formed by varying standard deviation of noiseadded to attributes of a plot associated with a decision tree orstrategy of providing offers.

A sweet spot 1408 can be determined on the tradeoff curve 1406. Thesweet spot 1408 can characterize theexploration-versus-exploitation-tradeoff. The location of the sweet spot1408 can be a position on the tradeoff curve 1406 where a substantialamount of information (as measured by the exploration index 1404) can begenerated, while profit 1402 may not be substantially decreased. Thus,the sweet spot 1408 can characterize a position where generated strategydata can be more than a first threshold while profit can be more than asecond threshold.

The term customer, as referred herein, can include a customer, anindividual, an entity, a person, personnel, and/or the like. The termoffer, as used herein, can include a discount, an allowance, acommission, a concession, an exemption, a coupon, a present, atime-share, a proposal, a presentation, cash, and/or any other offer.The term strategy, as used herein, can include a policy, a plan, anarrangement, intelligence, or any other strategy.

While two-dimensional graphs/plots have been described to show valuesfor two attributes, other models can exist when there are more than twoattributes. For example, when three attributes are used,three-dimensional models can be used; and when “n” attributes are used,n-dimensional models can be used.

Various implementations of the subject matter described herein may berealized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations may include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and may be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the term “machine-readable medium” refers toany computer program product, apparatus and/or device (e.g., magneticdiscs, optical disks, memory, Programmable Logic Devices (PLDs)) used toprovide machine instructions and/or data to a programmable processor,including a machine-readable medium that receives machine instructionsas a machine-readable signal. The term “machine-readable signal” refersto any signal used to provide machine instructions and/or data to aprogrammable processor.

To provide for interaction with a user, the subject matter describedherein may be implemented on a computer having a display device (e.g., aCRT (cathode ray tube) or LCD (liquid crystal display) monitor) fordisplaying information to the user and a keyboard and a pointing device(e.g., a mouse or a trackball) by which the user may provide input tothe computer. Other kinds of devices may be used to provide forinteraction with a user as well; for example, feedback provided to theuser may be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user may bereceived in any form, including acoustic, speech, or tactile input.

The subject matter described herein may be implemented in a computingsystem that includes a back-end component (e.g., as a data server), orthat includes a middleware component (e.g., an application server), orthat includes a front-end component (e.g., a client computer having agraphical user interface or a Web browser through which a user mayinteract with an implementation of the subject matter described herein),or any combination of such back-end, middleware, or front-endcomponents. The components of the system may be interconnected by anyform or medium of digital data communication (e.g., a communicationnetwork). Examples of communication networks include a local areanetwork (“LAN”), a wide area network (“WAN”), and the Internet.

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

Although a few variations have been described in detail above, othermodifications are possible. For example, the logic flow depicted in theaccompanying figures and described herein do not require the particularorder shown, or sequential order, to achieve desirable results. Otherembodiments may be within the scope of the following claims.

1-23. (canceled)
 24. A computing system connected to a communicationsnetwork for exchanging data, the computing system having at least oneprocessor, a non-transitory data storage medium, and a plurality of datastructures and executable code stored in the non-transitory storagemedium, the data structures including a first decision tree and a seconddecision tree, the execution of the executable code by the at least oneprocessor causing the computing system to: obtain data stored in atleast one data storage medium, the data corresponding to a plurality ofentities associated with a plurality of attributes; determine a firstoffer for a first attribute from among the plurality of attributes basedon the obtain data; form the first decision tree and the second decisiontree for characterizing one or more offers corresponding to the firstattribute; compare performance of the first decision tree withperformance of a second decision tree in relation to the firstattribute; provide a first offer to a first entity in accordance withthe first decision tree, in response to determining that the firstdecision tree is superior to the second decision tree in view of aperformance threshold; and provide a second offer to the first entity inaccordance with the second decision tree, in response to determiningthat the second decision tree is superior to the first decision tree inview of the performance threshold.
 25. The system of claim 24, wherein aplurality of causal models are utilized to form at least one of thefirst decision model and the second decision model to evaluate one ormore objectives of an entity, the causal model being used to determine abest offer for the first attribute.
 26. The system of claim 25, whereinthe causal models characterize a response of an entity to a historicaloffer.
 27. The system of claim 25, wherein the determining of the bestoffer is based on evaluation of at least one of a global maximum valueand a local maximum value by the first decision model and the seconddecision model.
 28. The system of claim 24, wherein the second decisiontree is obtained by changing a value of one or more attributesassociated with the first decision tree.
 29. The system of claim 24,wherein the performance of the first decision tree is characterized bybusiness efficacy provided by implementing a strategy associated withthe first decision tree, the business efficacy associated with the firstdecision tree being based on a plurality of iterations of strategyevolution, the business efficacy characterizes a profit of an entityproviding the offers to the plurality of entities.
 30. The system ofclaim 24, wherein the performance of the second decision tree ischaracterized by business efficacy provided by implementing a strategyassociated with the second decision tree, the business efficacyassociated with the second decision tree being based on a plurality ofiterations of strategy evolution, the business efficacy characterizes aprofit of an entity providing the offers to the plurality of entities.31. The system of claim 24, wherein the plurality of attributes arerepresented by a graph having a plurality of dots with at least one of acorresponding color and a corresponding intensity to characterize avalue of at least a first attribute for a first entity.
 32. The systemof claim 33, wherein the first attribute is modified by adding noise tothe first attribute by varying a standard deviation of a noisedistribution to determine the noise.
 33. The system of claim 32, whereinthe adding of the noise to the first attribute provides a first profitto a first entity providing the first offer, wherein the first profit ismore than a second profit obtained without the addition of the noise tothe first attribute.
 34. The system of claim 33, wherein the adding ofthe noise to the first attribute comprises adding Gaussian noise to thefirst attribute, wherein a standard deviation of the Gaussian noise isvaried to determine a tradeoff.
 35. The system of claim 34, wherein anupdated strategy associated with the tradeoff is generated and a secondoffer is provided based on the updated strategy being determined basedon a first tradeoff characterizing a balance between cost of a businessentity and a rate of updating the strategy.
 36. A computing systemconnected to a communications network for exchanging data, the computingsystem having at least one processor, a non-transitory data storagemedium, and a plurality of data structures and executable code stored inthe non-transitory storage medium, the data structures includingtransition probability table, the execution of the executable code bythe at least one processor causing the computing system to: determine aplurality of attributes associated with a strategy; change values of afirst set of one or more attributes in accordance with a transitionprobability table; vary one or more probabilities to determine a firsttradeoff; and generate, based on the first tradeoff, a first updatedstrategy.
 37. The system of claim 36, wherein the execution of theexecutable code by the at least one processor causes the computingsystem to: add noise to a second set of one or more attributes; varystandard deviation of a noise distribution associated with the noise todetermine a second tradeoff; and generate, based on the second tradeoff,a second updated strategy, the second updated strategy characterizing anupdate of the first updated strategy.
 38. The system of claim 37,wherein the execution of the executable code by the at least oneprocessor causes the computing system to provide, based on the secondupdated strategy, one or more offers to a plurality of entities.
 39. Thesystem of claim 38, wherein the first set of one or more attributesinclude gender and residential status.
 40. The system of claim 37,wherein the second set of one or more attributes include risk score andcredit line utilization.
 41. The system of claim 40, wherein based onthe transition probability table, eligibility constraints for providingthe one or more offers to the plurality of entities are determined andat least one of the first updated strategy and the second updatedstrategy are based on the eligibility constraints to exclude some offersto corresponding ineligible entities.
 42. The system of claim 41,wherein at least one of the first tradeoff and the second tradeoff aredetermined using corresponding tradeoff curves, at least one of thefirst tradeoff and the second tradeoff being characterized by asweet-spot on a corresponding tradeoff curve, the sweet-spotcharacterizing a position where generated strategy data is more than afirst threshold while profit is more than a second threshold.
 43. Thesystem of claim 42, wherein the plurality of attributes compriseobserved variables known at time of providing the offers, derivedvariables, predictive variables, and a score.
 44. The system of claim43, wherein the observed variables comprise data filled by entities inapplications, data associated with financial-accounts, demographicsdata, transaction data, credit bureau data, credit card score, creditcard usage data, risk score, revenue score, credit line utilizationdata, social network data, conversations of one or more entities withone of other entities and third parties, and third party data.
 45. Thesystem of claim 43, wherein the derived variables comprise textkeywords, n-grams, merger and acquisition transaction data, andparameters of social networks.
 46. The system of claim 43, wherein thepredictive variables comprise likelihood to default over a predeterminedperiod of time in future and expected entity lifetime value and thescore is calculated based on the observed variables.